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To comply with Section 1111(e) of the Elementary and Secondary Education Act (ESEA) as amended 
by the Every Student Succeeds Act (ESSA), states are required to submit evidence associated with their 
assessment systems to the United States Department of Education (ED) for peer review. Thirty-eight states 


submitted evidence for their assessment systems to the peer review process in 2016. As of May 2017, ED 


has issued thirty-seven decision letters in response. 


The peer review process requires cohorts of trained experts to evaluate state-submitted evidence as it 
relates to anumber of established criteria, organized by and divided into Critical Elements (CEs). Each 
decision letter outlines the CEs for which states must submit additional evidence to meet the statutory and 
regulatory requirements for standards and assessment based on peer review feedback on state-submitted 
documentation. 


This report will look at the CEs for which many states have been required to submit additional evidence 
across high school assessments for reading/language arts (R/LA) and mathematics, grades 3-8 
assessments for R/LA and mathematics, and alternate assessments. Lessons learned from high school and 
grades 3-8 science assessment feedback are also included. 


Peer Review of State Assessment Systems 

The assessment peer review process is in place to ensure that states implement rigorous academic standards and 
high-quality assessments. The process is evidence-based, focusing on documentation of the process used to develop 
and administer the assessments, and data to confirm the quality of the assessment system. The review is conducted 
by cohorts consisting of external assessment experts. Since ED paused peer review in 2012, almost every state has 
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developed and/or adopted new academic content standards and associated assessments. As a result, all states must 
submit documentation for peer review of R/LA, mathematics, and science general and alternate assessments. 


There are four possible outcomes for each CE of an assessment system: 

¢ Meets requirements - The component meets all of the requirements of the statute and regulations. The 
State is not required to submit any additional documentation for peer review unless significant changes are 
made to the State's standards or assessment. 

¢ Substantially meets requirements - The component meets most of the requirements of the statute and 
regulations. The State is required to submit some additional information, and the Department expects 
that this information should be able to be provided within one year. The State must submit a timeline for 
submission of the additional documentation, and the Department may take additional action if adequate 
progress is not made. 

¢ Partially meets requirements —- The component does not meet many of the requirements of the statute 
and regulations, and/or the State will need to submit substantial information to show that it meets the 
requirements. The Department expects that this information may not be able to be provided within one year, 
and a condition will be placed on the State's Title | grant award. A condition may also potentially be placed 
upon the State’s grant award under Part B of the Individuals with Disabilities Education Act (IDEA) if the matter 
pertains to requirements in the IDEA. The State will participate in regularly scheduled progress calls with ED, 
and, depending upon the component in question, the State may be placed on high-risk status and/or the 
State may be asked to enter into a compliance agreement. 

¢ Does not meet requirements - The component does not meet most of the requirements of the statute and 
regulations and will require substantial work by the State to revise. It is unlikely that the State will be able to 
make the necessary changes within one year, and may be required to halt administration of the assessment 
component in question. The State will participate in regularly scheduled progress calls with ED, and a 
condition will be placed on the State's Title | grant award. A condition may also potentially be placed upon the 
State’s grant award under Part B of IDEA if the matter pertains to requirements in the IDEA. Depending upon 
the component in question, the State may be placed on high-risk status and/or the State may be asked to 
enter into a compliance agreement.’ 


Summary of Peer Review Outcomes 
Each decision letter issued to a state provides detailed information for which assessments a state must provide 
additional evidence to meet the peer review requirements for a specific CE. 


The thirty-seven decision letters listed components that either substantially met requirements or partially met 
requirements. None of the submitted components were found to not meet requirements, per the decision letters 
issued. Not all assessment components are listed in all letters. In some cases, ED references a state’s entire assessment 
system in its request for additional evidence to meet the requirements of a CE. For the purposes of this report, this 
request for evidence has been applied to all assessments the state is required to administer, regardless of whether the 
assessment is specifically noted as having substantially or partially met requirements in the state's decision letter. 

: U.S. Department of Education Peer Review of State Assessment Systems: Non-Regulatory Guidance for States for Meeting Requirements of the Elementary and Secondary 


Education Act of 1965, as amended (1st ed.). (2015, September 25). Washington, DC: U.S. Department of Education, Office of Elementary and Secondary Education. 
Retrieved from https://www2.ed.gov/policy/elsec/guid/assessguid15.pdf. 


2 Whalen, A. (2016, October 6). Dear colleague state assessment system. [Letter from the U.S. Department of Education to the Council of Chief State School Officers, 
State Assessment Directors, State Title | Directors, and State Special Education Directors]. Retrieved from https://www2.ed.gov/admins/lead/account/saa/ 
dcletterassepeerreview1072016ltr.pdf. 
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Not all assessments are included in decision letter feedback, as the requirements may have been met, or, the 


assessment may not have been submitted for review. 


A visualization of each assessment component and its compliance level of 37 out of the 38 submissions has been 


created below: 
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After an analysis across all the decision letters issued, it was determined that a number of states were required to 
submit additional evidence in reference to like CEs. It is worth noting, however, that the evidence requested for the 
same CE across the decision letters sometimes differed. This report will analyze a sample of the CEs most widely cited 
as needing additional evidence submitted for general assessments in high school R/LA and mathematics, grades 3-8 
R/LA and mathematics, alternate assessments across content areas and grade levels, and high school and grades 3-8 
science assessments. 


High School R/LA and Mathematics 


CE 2.1 - Test Design and Development 


¢ Appears in 32 Decision Letters for High School R/LA and Mathematics Assessments 


CE 2.1 requires states to provide evidence that “the State’s test design and test development process is well-suited for 
the content, is technically sound, aligns the assessments to the full range of the State’s academic content standards, 
and includes: 


¢ Statements(s) of the purposes of the assessments and the intended interpretations and uses of results; 


¢ Test blueprints that describe the structure of each assessment in sufficient detail to support the development 
of assessments that are technically sound, measure the full range of the State’s grade-level academic content 
standards, and support the intended interpretations and uses of the results; 


¢ Processes to ensure that each assessment is tailored to the knowledge and skills included in the State's 
academic content standards, reflects appropriate inclusion of challenging content, and requires complex 
demonstrations or applications of knowledge and skills (i.e., higher-order thinking skills); 


¢ Ifthe State administers computer-adaptive assessments the item pool and item selection procedures 
adequately support the test design.” 


The additional evidence requested to be submitted for CE 2.1 for high school R/LA and mathematics is varied, but the 
feedback provided to consortium states is generally consistent across decision letters. 


Peer Review Feedback 


The feedback given for all Smarter Balanced consortium states that submitted evidence for initial review requires 
them to submit evidence for CE 2.1 for high school R/LA and mathematics to show that the “test design aligns the 
assessments to the full depth and breadth” of the content standards, “that item selection procedures for computer 
adaptive test online assessment adequately deliver tests that meet test design requirements for the intended depth 
of knowledge (DOK) of the assessments,’ “that...assessments produce grade level student achievement scores that are 
based only on grade-level content items,” and “that the item pools for all versions of the assessments...are sufficient to 
support the test design requirements” (Smarter Balanced Decision Letters). California was also asked for “clarification 


as to whether (and, if so, how) CDE has incorporated its supplemental content standards into the test design and 
development process for its assessments.” 


All PARCC states that submitted evidence for initial review were asked to provide additional “evidence that the 
assessment design measures the full breadth and depth of the state’s academic content standards” (PARCC Decision 
Letters). 


3 U.S. Department of Education Peer Review of State Assessment Systems: Non-Regulatory Guidance for States for Meeting Requirements of the Elementary and Secondary 
Education Act of 1965, as amended (1st ed.). (2015, September 25). Washington, DC: U.S. Department of Education, Office of Elementary and Secondary Education. 
Retrieved from https://www2.ed.gov/policy/elsec/guid/assessguid15.pdf. 
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States administering the ACT as their assessment for accountability purposes were asked to submit “evidence in 
reference to the State’s plan to assess the full breadth of the State’s R/LA standards” for CE 2.1 (WI, WY Decision Letters). 


For a variety of reasons, states that administer state-specific assessments were required to submit additional evidence 
for CE 2.1, including “evidence that describes how cognitive complexity is used for test construction” (NC Decision 
Letter). In the cases of Missouri and Florida's R/LA assessment, the additional evidence requested was the same as that 


requested of PARCC consortium states. 


Some letters include examples of evidence that states may wish to consider submitting to meet the panel's 
requirements for this CE. For instance, Kentucky was given the example of submitting “an independent alignment 
study” to provide evidence that “test blueprints align to the full range of academic content standards.’ States are not 
expected to submit additional evidence surrounding speaking and listening if they have received a waiver. 


Submission of Additional Evidence Requested 


Based on ED’s Map of the Critical Elements (see Appendix), CE 2.1 is “likely addressed by coordinated evidence,” so 
states in a consortium may wish to coordinate efforts to submit the additional evidence requested. However, states 
must also submit evidence specific to their state’s standards/assessments. 


Looking at the decision letters received by other states may provide some insight as to the types of additional 
evidence that states may choose to submit. The Guidance also provides examples of evidence states may submit for 
CE 2.1 to address issues, but “a State may determine that other types of evidence better address a critical element.” 


A sampling of examples of evidence from the Guidance that may address CE 2.1 include: 
¢ Relevant sections of State or regulations, language from contract(s) for the State’s assessments, test 
coordinator or test administrator manuals, or other relevant documentation that states the purposes of the 
assessment and the intended interpretations and uses of results; 


¢ Test blueprints that: 


> Describe the structure of each assessment in sufficient detail to support the development of a technically 
sound assessment, for example, in terms of the number of items, item types, the proportion of item types, 
response formats, the range of item difficulties, the types of scoring procedures, and applicable time limits; 


>» Align to the State’s grade-level academic content standards in terms of content (i.e., knowledge and 
cognitive process), the full range of the State’s grade-level academic content standards, the balance of 
content, and cognitive complexity; 


¢ Documentation that the test design is tailored to the specific knowledge and skills in the State’s academic 
content standards (e.g., includes extended response items that require demonstration of writing skills if the 
State’s reading/language arts academic content standards include writing). 


See the Guidance document for a full list of potential examples of evidence that may address CE 2.1. 


Refer to the Center on Standards and Assessment Implementation (CSAI) Peer Review Assistance document for more 


information on how CSAI can provide support to your state throughout the peer review process. 
CE 5.4 - Monitoring Test Administration for Special Populations 


¢ Appears in 25 Decision Letters for High School R/LA and Mathematics Assessments 


CE 5.4 requires states to submit evidence that “the state monitors test administration in its districts and schools to 
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ensure that appropriate assessments, with or without appropriate accommodations, are selected for students with 
disabilities under IDEA, students covered by Section 504, and English learners so that they are appropriately included 
in assessments and receive accommodations that are: 


¢ Consistent with the State’s policies for accommodation; 
¢ Appropriate for addressing a student's disability or language needs for each assessment administered; 
¢ Consistent with accommodations provided to students during instruction and/or practice; 


¢ Consistent with the assessment accommodations identified by a student’s IEP team or 504 team for students 
with disabilities, or another process for an English learner; 


¢ Administered with fidelity to test administration procedures.”* 


This CE will be checked for completeness by ED, and for consortium states, is likely to be addressed by a mix of state 
and coordinated evidence per the Map of the Critical Elements. 


Peer Review Feedback 


All states that are required to submit additional evidence for this CE are required to do so, not just in relation to high 
school R/LA and mathematics, but also for additional portion(s) or the entirety of their state assessment systems. 
This is the case across all types of assessments that a state may give (Smarter Balanced, PARCC, ACT, or state-specific 
assessments). 

States should be aware that the type of evidence submitted is important to the review process. The documentation 
submitted to ED should include procedural evidence (e.g., documentation of procedures and protocols, studies that 
indicate how a particular feature or part of an assessment will function), as well as confirmatory evidence (e.g., proof 
that procedures and protocols were followed, analyses that indicate that a particular feature or part of an assessment 
functioned as intended). In the case of CE 5.4, states may wish to submit procedural documentation regarding test 
administration (e.g., protocols, etc.), in addition to evidence that the prescribed monitoring protocols were followed 
during the assessment event.> 


Submission of Additional Evidence Requested 
The Guidance document provides examples of evidence states may choose to submit to support their assertion that 
they meet CE 5.4. 
“Evidence to support this critical element for the State’s assessment system includes documents such as: 
¢ Description of procedures the State uses to monitor that accommodations selected for students with 


disabilities, students covered by Section 504, and English learners are appropriate; 


¢ Description of procedures the State uses to monitor that students with disabilities are placed by IEP Teams in 
the appropriate assessment; 


¢ The State's written procedures for monitoring the use of accommodations during test administration, such as 
guidance provided to districts; instructions and protocols for State, district and school staff; and schedules for 
monitoring; 


¢ Summary of results of monitoring for the most recent year of test administration in the State.” (p. 48) 
4 U.S. Department of Education Peer Review of State Assessment Systems: Non-Regulatory Guidance for States for Meeting Requirements of the Elementary and Secondary 


Education Act of 1965, as amended (1st ed.). (2015, September 25). Washington, DC: U.S. Department of Education, Office of Elementary and Secondary Education. 
Retrieved from https://www2.ed.gov/policy/elsec/guid/assessguid15.pdf. 


° U.S. Department of Education State Assessment Peer Review Update. (2017, February 23). [PowerPoint slides]. Washington, DC: U.S. Department of Education. 
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The type(s) of documentation a state will choose to submit to fulfill the request for additional evidence should be 
guided by the specific feedback and comments provided to the state from peer reviewers and ED. Unlike CE 2.1 
detailed above, the requests for additional evidence to be submitted for CE 5.4 cannot be compartmentalized by 
assessment system (i.e., ACT, PARCC, Smarter Balanced, or state-specific). States must look to their specific feedback to 
provide the documentation necessary to address the requests of the peer reviewers and ED. Examples of the types of 
documentation requested include: 


¢ Evidence that test monitoring procedures and training address test accommodations and fidelity to test 
administration procedures (AL Decision Letter); 


¢ Evidence that [the state] monitors test administrations for special populations for consistency among 
individualized education programs, classroom instruction, and accommodations (DC Decision Letter, Rl 


Decision Letter); 


¢ Evidence that the state monitors test administration in districts and schools to ensure that appropriate 
assessments and accommodations are selected for students with disabilities under IDEA, students covered by 
Section 504, and English learners (IA Decision Letter, WV Decision Letter). 


Grades 3-8 R/LA and Mathematics 
CE 3.1 - Overall Validity, Including Validity Based on Content 


¢ Appears in 33 Decision Letters for Grades 3-8 R/LA and Mathematics Assessments 


CE 3.1 asks states to submit evidence that shows “adequate overall validity evidence for its assessments, and the State’s 
validity evidence includes evidence that the State’s assessments measure the knowledge and skills specified in the 
State’s academic content standards, including: 
¢ Documentation of adequate alignment between the State’s assessments and the academic content standards 
the assessments are designed to measure in terms of content (i.e., knowledge and process), the full range of 
the State’s academic content standards, balance of content, and cognitive complexity; 


¢ Ifthe State administers alternate assessments based on alternate academic achievement standards, the 
assessments show adequate linkage to the State’s academic content standards in terms of content match (i.e., 
no unrelated content) and the breadth of content and cognitive complexity determined in test design to be 
appropriate for students with the most significant cognitive disabilities.’ 


This analysis is only considering validity evidence surrounding the assessments for grades 3-8 in reading/language 
arts and mathematics, and is not looking at validity evidence for a state’s alternate assessments based on alternate 
academic achievement standards. 


On ED’s Map of the Critical Elements, CE 3.1 is shown as likely to be addressed by coordinated evidence. 


Peer Review Feedback 


PARCC states that did not meet the documentation requirements for this CE have all been requested to submit 
“additional alignment evidence that addresses all remaining grades (grades 3, 4, 6, and 7) in both R/LA and 
mathematics content areas.’ With the exception of Maryland, the entire group must also submit “alignment evidence 
that supports a test design that assesses the full range of the State's academic content standards for all tested grades” 
(PARCC Decision Letters). 
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States in the Smarter Balanced consortium required to submit additional evidence for CE 3.1 were all asked to provide: 


¢ Evidence as noted for all item pools in element 2.1; 


¢ Evidence that Smarter Balanced assessments that include off-grade-level content conform to the on-grade- 
level blueprint for the assessment; 


¢ Evidence of alignment of sample test forms for grades 3, 4, 6, and 7 in R/LA and mathematics; 


¢ Evidence of improved alignment of the (Smarter Balanced Decision Letters). 


North Dakota and Vermont, members of the Smarter Balanced consortium, were also asked to provide “evidence of a 
summary report that the CAT administered test forms matched test blueprints” (ND, VT Decision Letter). 


Submission of Additional Evidence Requested 


Both of these groups of consortium states were required to submit alignment evidence across grades 3, 4, 6, and 
7 in R/LA and mathematics. Submission of alignment evidence at grades 5 and 8 is not sufficient for meeting ED’s 
requirements for CE 3.1, and states in these consortia will have to go back to their respective test developers and 
request that they compile the stated alignment evidence. 


States administering non-consortia assessments were asked to submit a range of additional evidence and 
documentation for CE 3.1 (AL, AZ, FL, IA, IN, KS, KY, MN, NC, NY, PA, TX, UT, VA, WY Decision Letters). Many statements 
of additional evidence requested are tied to ED’s requests for additional evidence of CE 2.1, as well as evidence of 
alignment or improved alignment based upon previously submitted evidence. 


CE 6.4 — Reporting 

¢ Appears in 25 Decision Letters for Grades 3-8 R/LA and Mathematics Assessments 
CE 6.4 asks states to submit evidence that “the State reports its assessment results, and the reporting facilitates timely, 
appropriate, credible, and defensible interpretations and uses of results for students tested by parents, educators, 
State officials, policymakers and other stakeholders, and the public, including: 

¢ The State reports to the public its assessment results on student achievement at each proficiency level and 


the percentage of students not tested for all students and each student group after each test administration; 


¢ The State reports assessment results, including itemized score analyses, to districts and schools so that 
parents, teachers, principals, and administrators can interpret the results and address the specific academic 
needs of students, and the State also provides interpretive guides to support appropriate uses of the 
assessment results; 


¢ The State provides for the production and delivery of individual student interpretive, descriptive, and 
diagnostic reports after each administration of its assessments that: 


> Provide valid and reliable information regarding a student's achievement; 


> Report the student's achievement in terms of the State’s grade-level academic achievement standards 
(including performance-level descriptors); 


> Provide information to help parents, teachers, and principals interpret the test results and address the 
specific academic needs of students; 


> Are available in alternate formats (e.g., Braille or large print) upon request and, to the extent practicable, in 
a native language that parents can understand; 
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¢ The State follows a process and timeline for delivering individual student reports to parents, teachers, and 
principals as soon as practicable after each test administration.” 


It has been indicated on the Map of the Critical Elements that CE 6.4 is likely addressed with a mix of coordinated (for 
consortium states) and state-specific evidence, and will also be checked for completeness by ED. 


Peer Review Feedback 


Though listed in this report under R/LA and mathematics for grades 3-8, the feedback requested for CE 6.4 has been 
requested across multiple areas of the affected states’ assessment systems, regardless of the type of assessment 
administered. However, the type(s) of evidence requested are varied across assessments. 


For states that need to submit additional documentation for this CE and also administer the Smarter Balanced 
assessment, evidence that individual student score reports are available in alternate formats (upon request) must 
be provided (CA, CT, ID, OR, SD, VT, WA Decision Letters). With the exception of Idaho and Vermont, documentation 
regarding the process and timeline for delivery of individual score reports to parents, teachers, and principals must 


also be submitted. 


PARCC consortium states required to submit additional evidence have been asked to provide documentation 
surrounding multiple components of CE 6.4. Some states were asked to provide the same evidence as the affected 
Smarter Balanced states, and others were also asked to provide other additional evidence in order to meet the 
requirements of this CE. The documentation requested includes evidence that state departments of education and 
PARCC “provide reports that enable itemized score analyses to local educational agencies and schools” (CO, IL, MD, NJ, 
NM, RI Decision Letters). The District of Columbia is required to provide “evidence that all information in [state] reports 
of PARCC assessments support the same interpretations as those in the PARCC-produced reports” (DC Decision Letter). 


States that administer state-specific assessments were required to submit much of the same additional evidence as 
Smarter Balanced and PARCC consortium states. Additionally, states were required to provide “examples of state-level 
reports that include the percentage of students not tested as required for this element” (AL Decision Letter), “evidence 


of current individual student reports that shows student achievement in terms of State grade-level achievement 
standards” (FL Decision Letter), and “evidence that performance or achievement level descriptors are included with 
individual score reports” (TX Decision Letter). 


Submission of Additional Evidence Requested 


The Guidance document explains that the evidence and supporting documentation states choose to submit to ED 

for review of CE 6.4 must “demonstrate that the State’s reporting system facilitates timely, appropriate, credible, and 
defensible interpretation and use of its assessment results.” However, states must also protect the privacy of test 
taskers when this information is submitted. Samples “should be redacted to protect personally identifiable information 
... of populated with information about a fictitious student for illustrative purposes.’ To support this CE, a state may 
provide the same types of evidence for grades 3-8 and high school general assessments, as well as for alternate 


assessments administered (if required based on feedback). 
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Alternate Assessments (R/LA, Mathematics, Science) 


Alternate Assessments (R/LA, Mathematics, Science) 

Note: Some states were required to submit additional evidence for CEs across different combinations of 
assessment types within their assessment systems. CE 4.1 and CE 5.1 were chosen to be examined specifically 

for the Alternate Assessment portion of this brief because they had some of the highest occurrences of needing 
additional evidence submitted for a state’s alternate assessment suite. States that were given feedback with regard 
to their “entire assessment system” for a particular CE have been included. 


CE 4.1 — Reliability 


¢ Appears in 27 Decision Letters for Alternate Assessments 


CE 4.1 requires that “the State has documented adequate reliability evidence for its assessments for the following 
measures of reliability for the State’s student population overall and each student group, and if the State’s assessments 
are implemented in multiple States, for the assessment overall and each student group, including: 


¢ Test reliability of the State’s assessments estimated for its student population; 
¢ Overall and conditional standard error of measurement of the State’s assessments; 


¢ Consistency and accuracy of estimates in categorical classification decisions for the cut scores and 
achievement levels based on the assessment results; 


¢ For computer-adaptive tests, evidence that the assessments produce test forms with adequately precise 
estimates of a student's achievement.” 


Coordinated evidence likely addresses CE 4.1 for consortium states. 


Though use of the word “adequate” in the description of CE 4.1 does not provide an explicit level of detail of the 
volume of evidence that will meet ED’s requirements, the Guidance document does provide States with examples of 
evidence they may wish to provide. 


Peer Review Feedback 


The decision letters sent to states that contained requests for additional documentation and evidence for CE 4.1 for 
alternate assessments included feedback for R/LA, mathematics, and science content areas. Not all states are required 
to submit additional evidence for all subject areas/grade levels for their alternate assessments. Decision letters indicate 
for which components of a state’s alternate assessment suite additional evidence is required. 


States that submitted evidence to peer review, administer the National Center and State Collaborative Multi-State 
Alternate Assessment (NCSC/MSAA), and are requested to submit additional evidence for CE 4.1 were provided with 
the same feedback, directing them back to the explanatory note(s) already provided for writing items in CE 3.1 - 
Overall Validity, including Validity Based on Content. CE 3.1 has been analyzed previously in this report in the context 
of a state’s grades 3-8 general assessments, but many of the same types of evidence submitted to address CE 3.1 for 
the NCSC/MSAA can also be used to provide additional evidence to support a state's claim that it meets requirements 
for CE 4.1 (AR, AZ, CT, DC, ID, NM, RI, SD Decision Letters). 


States that administer the Dynamic Learning Maps Integrated (DLM-Int) assessment model for their alternate R/LA 
and mathematics assessments are also directed back to feedback provided for CE 3.1, and are asked specifically for 
“evidence of model fit analysis.” These states are also required to submit “evidence regarding consistency and accuracy 
of classifications, which will be satisfied by the response to element 3.3.’ CE 3.3 - Validity Based on Internal Structure is 
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likely addressed by coordinated evidence, and requires that a state “has documented adequate validity evidence that 
the scoring and reporting structures of its assessments are consistent with the sub-domain structures of the State’s 
academic content standards on which the intended interpretations and uses of results are based” (IA, KS, MO, ND, UT, 
VT Decision Letters). 


States administering the Dynamic Learning Maps Year End (DLM-YE) model for R/LA and mathematics are required 
to submit “evidence of monitoring and refinement of the diagnostic classification models from subsequent test 
administrations” (CO, IL, NJ, WI, WV Decision Letters). 


State-specific alternate assessments that must submit additional evidence to meet the requirements of CE 4.1 must 
submit a variety of documentation across content areas and grade levels (available in ED’s comments to the state in 
the decision letter issued). The documentation requested includes “reliability estimates for all student sub-groups, 
where feasible to do so given minimum n-size” (HI Decision Letter), “evidence of...reliability estimates by gender and 


dichotomous demographic categories to demonstrate lack of bias” (DE Decision Letter), and “evidence of...inter-rater 


reliability” (WY Decision Letter), among others. 


CE 5.1 - Procedures for Including Students with Disabilities 


¢ Appears in 21 Decision Letters for Alternate Assessments 


CE 5.1 - Procedures for Including Students with Disabilities requires states to submit evidence that it “has in place 
procedures to ensure the inclusion of all public elementary and secondary school students with disabilities in the 
State’s assessment system, including, at a minimum, guidance for individual educational program (IEP) teams to inform 
decisions about student assessments that: 
¢ Provides clear explanations of the differences between assessments based on grade-level academic 

achievement standards and assessments based on alternate academic achievement standards, including 

any effects of State and local policies on a student's education resulting from taking an alternate assessment 

based on alternate academic achievement standards; 


¢ States that decisions about how to assess students with disabilities must be made by a student’s IEP Team 
based on each student's individual needs; 


¢ Provides guidelines for determining whether to assess a student on the general assessment without 
accommodation(s), the general assessment with accommodation(s), or an alternate assessment; 


¢ Provides information on accessibility tools and features available to students in general and assessment 
accommodations available for students with disabilities; 


¢ Provides guidance regarding selection of appropriate accommodations for students with disabilities; 


¢ Includes instructions that students eligible to be assessed based on alternate academic achievement 
standards may be from any of the disability categories listed in the IDEA; 


¢ Ensures that parents of students with the most significant cognitive disabilities are informed that their 
student's achievement will be based on alternate academic achievement standards and of any possible 
consequences of taking the alternate assessments resulting from district or State policy (e.g., ineligibility for a 
regular high school diploma if the student does not demonstrate proficiency in the content area on the State's 
general assessments); 


¢ The State has procedures in place to ensure that its implementation of alternate academic achievement 
standards for students with the most significant cognitive disabilities promotes student access to the general 
curriculum.” 
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CE 5.1 contains many components, and sufficient evidence must be submitted for each to meet the peer review 
requirements and comply with ESSA. 


Peer Review Feedback 


The additional evidence these states are required to submit includes: 
¢ Evidence including guidelines for IEP teams to use in determining whether to assess a student on the general 
assessment without accommodation(s), the general assessment with accommodation(s), or an alternate 
assessment (IA, KY Decision Letters); 


¢ Evidence that [the state] provides clear explanations of the differences between assessments based on grade- 
level academic achievement standards and assessments based on alternate achievement standards (AR, AZ, 
KS, KY, MO, WI Decision Letters); 


¢ Evidence that sufficient information is provided to parents of students with the most significant cognitive 
disabilities, and that they are informed that their student’s achievement will be based on alternate academic 
achievement standards and/or are informed of any possible consequences/implications of taking the AA- 
AAAS (e.g., ineligibility for a regular high school diploma if the student does not demonstrate proficiency in 
the content area on the State’s general assessments) (AR, CO, CT, IL, KS, MO, NJ, OR, SD, TX, VT, WY Decision 
Letters). 


The additional evidence requested is not delineated by assessment type. States that administer the same alternate 
assessments (e.g., DLM-YE, DLM-INT, or NCSC/MSAA) may be required to submit different pieces of additional 
evidence, most likely because much of the documentation needed to meet the requirements of CE 5.1 consists of 
communication and protocols at the state and district levels. Additionally, the Map of the Critical Elements indicates 
that CE 5.1 is “likely addressed by state-specific evidence.” 


Submission of Additional Evidence Requested 


Examples of evidence to support CE 5.1 are provided in the Guidance document. However, based on the feedback they 
receive from ED, states may have to submit additional documentation not listed in the examples. This may include 
evidence of stakeholder communication that outlines the key differences between assessments based on grade-level 
academic standards and those based on alternate achievement standards, documentation of evidence provided to 
parents of students with the most significant cognitive disabilities, and evidence of the procedures and guidelines 

in place for use by individualized education plan (IEP) teams to determine which assessment/accommodations are 
needed by each student. 


Science Assessments 


Of the thirty-seven decision letters issued, thirteen explicitly reference whether high school science assessments 
partially or substantially meet requirements (CO, DE, FL, HI, IA, ID, KY, MN, NC, TX, UT, VA, WY Decision Letters), and 
twelve explicitly reference whether grades 3-8 science assessments partially or substantially meet requirements (AL, 
CO, DE, FL, HI, IA, MN, MO, NC, OH, TX, UT, VA Decision Letters). 


Other decision letters issued to states did not explicitly mention science assessments, but some of the feedback 
provided to states for specific CEs reference the entire assessment system. 
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Of the letters that do reference science assessments directly as either partially or substantially meeting requirements, 
the CEs that were cited most often as in need of additional documentation include CEs 2.1, 3.1, 4.2, 5.1, and 5.4 for 
high school science assessments, and CEs 2.1, 3.1, 5.4, and 6.4 for science assessments at grades 3-8. A full description 
of each CE can be found below: 


High School Science Assessment CEs Most Cited for Assessments That Partially/Substantially Meet Requirements: 


¢ CE2.1-Test Design and Development 


The State's test design and test development process is well-suited for the content, is technically sound, aligns 
the assessments to the full range of the State’s academic content standards, and includes: 


> Statement(s) of the purposes of the assessments and the intended interpretations and uses of results; 


> Test blueprints that describe the structure of each assessment in sufficient detail to support the 
development of assessments that are technically sound, measure the full range of the State’s grade-level 
academic content standards, and support the intended interpretations and uses of the results; 


> Processes to ensure that each assessment is tailored to the knowledge and skills included in the State’s 
academic content standards, reflects appropriate inclusion of challenging content, and requires complex 
demonstrations or applications of knowledge and skills (i.e., higher-order thinking skills); 


> Ifthe State administers computer-adaptive assessments, the item pool and item selection procedures 
adequately support the test design. 


¢ CE 3.1 - Overall Validity, Including Validity Based on Content 
The State has documented adequate overall validity evidence for its assessments, and the State's validity 
evidence includes evidence that the State’s assessments measure the knowledge and skills specified in the 
State’s academic content standards, including: 


> Documentation of adequate alignment between the State’s assessments and the academic content 
standards the assessments are designed to measure in terms of content (i.e., knowledge and process), the 
full range of the State’s academic content standards, the balance of content, and cognitive complexity; 


> Ifthe State administers alternate assessments based on alternate academic achievement standards, the 
assessments show adequate linkage to the State’s academic content standards in terms of content match 
(i.e., no unrelated content) and the breadth of content and cognitive complexity determined in test design 
to be appropriate for students with the most significant cognitive disabilities. 


¢ CE 4.2 - Fairness and Accessibility 
The State has taken reasonable and appropriate steps to ensure that its assessments are accessible to all 
students and fair across student groups in the design, development, and analysis of its assessments. 


¢ CE5.1- Procedures for Including Students with Disabilities 
The State has in place procedures to ensure the inclusion of all public elementary and secondary school 
students with disabilities in the State’s assessment system, including, at a minimum, guidance for individual 
educational plan (IEP) teams to inform decisions about student assessments that: 


> Provides clear explanations of the differences between assessments based on grade-level academic 
achievement standards and assessments based on alternate academic achievement standards, including 
any effects of State and local policies on a student’s education resulting from taking an alternate 
assessment based on alternate academic achievement standards; 
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> States that decisions about how to assess students with disabilities must be made by a student's IEP Team 
based on each student's individual needs; 


> Provides guidelines for determining whether to assess a student on the general assessment without 
accommodation(s), the general assessment with accommodation(s), or an alternate assessment; 


> Provides guidelines for determining whether to assess a student on the general assessment without 
accommodation(s), the general assessment with accommodation(s), or an alternate assessment; 


> Provides information on accessibility tools and features available to students in general and assessment 
accommodations available for students with disabilities; 


> Provides guidance regarding selection of appropriate accommodations for students with disabilities; 


> Includes instructions that students eligible to be assessed based on alternate academic achievement 
standards may be from any of the disability categories listed in the IDEA; 


> Ensures that parents of students with the most significant cognitive disabilities are informed that their 
student's achievement will be based on alternate academic achievement standards and informed of any 
possible consequences of taking the alternate assessments resulting from district or State policy (e.g., 
ineligibility for a regular high school diploma if the student does not demonstrate proficiency in the 
content area on the State’s general assessments); 


> The State has procedures in place to ensure that its implementation of alternate academic achievement 
standards for students with the most significant cognitive disabilities promotes student access to the 
general curriculum. 


¢ CE5.4- Monitoring Test Administration for Special Populations 
The State monitors test administration in its districts and schools to ensure that appropriate assessments, 
with or without appropriate accommodations, are selected for students with disabilities under IDEA, students 
covered by Section 504, and English learners so that they are appropriately included in assessments and 
receive accommodations that are: 


> Consistent with the State's policies for accommodations; 

> Appropriate for addressing a student's disability or language needs for each assessment administered; 
> Consistent with accommodations provided to the students during instruction and/or practice; 
> 


Consistent with the assessment accommodations identified by a student's IEP team or 504 team for 
students with disabilities, or another process for an English learner; 


>» Administered with fidelity to test administration procedures. 


Grades 3-8 Science Assessment CEs Most Cited for Assessments That Partially/Substantially Meet Requirements: 

¢ CE2.1-Test Design and Development (see CE 2.1 above) 

¢ CE 3.1 - Overall Validity, Including Validity Based on Content (see CE 3.1 above) 

¢ CE5.4- Monitoring Test Administration for Special Populations (see CE 5.4 above) 

* CE6.4 - Reporting 
The State reports its assessment results, and the reporting facilitates timely, appropriate, credible, and 
defensible interpretations and uses of results for students tested by parents, educators, State officials, 
policymakers and other stakeholders, and the public, including: 


> The State reports to the public its assessment results on student achievement at each proficiency level and 
the percentage of students not tested for all students and each student group after each test administration; 
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> The State reports assessment results, including itemized score analyses, to districts and schools so that 
parents, teachers, principals, and administrators can interpret the results and address the specific academic 
needs of students, and the State also provides interpretive guides to support appropriate uses of the 
assessment results; 


> The State provides for the production and delivery of individual student interpretive, descriptive, and 
diagnostic reports after each administration of its assessments that: 


= Provide valid and reliable information regarding a student's achievement; 


= Report the student’s achievement in terms of the State’s grade-level academic achievement standards 
(including performance-level descriptors); 


= Provide information to help parents, teachers, and principals interpret the test results and address the 
specific academic needs of students; 


= Are available in alternate formats (e.g., Braille or large print) upon request and, to the extent practicable, 
in a native language that parents can understand; 


>» The State follows a process and timeline for delivering individual student reports to parents, teachers, and 
principals as soon as practicable after each test administration. 
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Critical elements in clear boxes with solid outlines likely will be addressed with 
State-specific evidence, even if a State administers the same assessments 


administered by other States (e.g. Critical Element 5.1). 


« Critical elements in ovals or clear boxes with dashed outlines likely will be 


addressed by both State-specific evidence and coordinated evidence for States 
administering the same assessments (e.g. Critical Element 2.3, 5.4). 
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